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SYSTEM AND METHOD FOR DISPLAYING AN IMAGE STREAM 

FIELD OF THE INVENTION 

The present invention relates to a method and system for displaying and/or 
5 reviewing image streams. 

BACKGROUND OF THE INVENTION 

An image stream may be assembled from a series of still images and 
displayed to a user. The images may be created or collected from various 

10 sources. For example, a known in-vivo imaging system which includes a 
swallowable capsule, may, for example, capture images of a lumen such as the 
gastrointestinal (Gl) tract and may transmit them to an external recording device. 
Large numbers of images may be collected for viewing and, for example, combined 
in sequence. An image stream of, for example, 40 minutes in length, containing for 

15 example about 4,800 frames, may be available to, for example a health 
professional for review. Other numbers of frames or lengths may be used. In one 
embodiment, a health professional may, for example, use the images to diagnose 
pathological conditions of the Gl tract, and, in addition, the system may provide 
information about the location of these pathologies. Pathology, if present may 

20 typically be found in only a few locations along the long traverse of the Gl tract, and 
therefore in a small percentage of the recorded images. 

In general, if possible, a user may try to set the streaming rate to the 
highest rate where the user can quickly review the image stream without missing 
important information that may be present in any of the images included in the 

25 stream. The rate at which a user can effectively review an image stream may be 
limited by a physiological averaging above which certain details in individual 
images displayed in the stream may be physiologically filtered out. 

SUMMARY OF THE INVENTION 

30 The system and method according to some embodiments of the present 

invention may allow an image stream to be viewed while emphasizing images that 
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may be of interest (e.g., images of pathologies). Tiius, a user may quioi<ly locate 
selected images of interest out of a lengthy image stream and an efficient and 
short view time may be provided. Other uses or benefits of an image display 
method according to some embodiments of the invention may exist. 

5 In one embodiment, a system and method are provided for displaying an 

image stream, such that multiple images from the image stream are displayed 
substantially simultaneously in a time slot Substantially simultaneously may 
include simultaneously. As such a single-frame stream display may be converted 
to a multi-frame stream display that may be shorter in viewing time. In a further 

10 embodiment of the invention, the spatial position of the images displayed 
substantially simultaneously in each time slot may be variably adjusted based on a 
predetennined criterion. For example, the spatial order or position of the images 
displayed substantially simultaneously in each time slot may correspond to a 
degree of variation of each image compared to a selected or generated reference 

15 image. In a further example, images displaying pathological looking tissue may be 
positioned away from images displaying healthy looking tissue. In one 
embodiment of the invention, images from a group of images displayed 
substantially simultaneously, containing a high degree of variation as compared to 
the other images in the group may be displayed in a spatial position where a viewer 

20 may typically draw his/her eyes first. In certain embodiments, the images may be 
collected from an autonomous capsule traversing the Gl tract. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention will be understood and appreciated more fully from 
25 the following detailed description taken in conjunction with the drawings in which: 

Figure 1A shows a schematic diagram of an in-vivo imaging system 
according to one embodiment of the present invention; 

Figure 1B shows a schematic diagram of a monitor display according to 
one embodiment of the present invention; 
30 Figure 2 shows a portion of a display according to an embodiment of ttie 

present invention; 
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Figure 3 shows a flow chart describing a method for spatially positioning a 
plurality of image frames displayed substantially simultaneously, according to an 
embodiment of the present invention; 

Figure 4 shows a flow chart describing a method for spatially positioning a 
5 plurality of image frames displayed substantially simultaneously, according to 
another embodiment of the present invention; 

Figure 5 shows a portion of a display according to a second embodiment 
of the present invention; 

Figure 6 shows an exemplary flow chart describing a method for combining 
10 multiple image streams into a single image stream according to an embodiment of 
the present invention; and 

Figure 7 showing an exemplary flow chart for adjusting the stream rate of 
the multi-frame stream according to one embodiment of the present invention. 

It will be appreciated that for simplicity and clarity of illustration, elements 
15 shown in the figures have not necessarily been drawn accurately or to scale. For 
example, the dimensions of some of the elements may be exaggerated relative to 
other elements for clarity, or several physical components may be included in one 
functional block or element. Further, where considered appropriate, reference 
numerals may be repeated among the figures to indicate corresponding or 
20 analogous elements. 
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DETAILED DESCRIPTION OF THE INVENTION 

In the following description, various aspects of the present invention will be 
described. For purposes of explanation, specific configurations and details are set 
5 forth in order to provide a thorough understanding of the present invention. 
However, it will also be apparent to one skilled in the art that the present invention 
may be practiced without the specific details presented herein. Furthermore, well- 
known features may be omitted or simplified in order not to obscure the present 
invention. 

10 Reference is made to Fig. 1A, which shows a schematic diagram of an in- 

vivo imaging system according to one embodiment of the present invention. In an 
exemplary embodiment, the system may include an imaging device 40, such as for 
example a capsule having an image sensor 46, for capturing images, an 
illumination source 42, for illuminating the body lumen, a transmitter 41, and an 

15 antenna 44, for transmitting images and possibly other information to, for example, 
a receiving device. Typically, the imaging device may correspond to embodiments 
described in U.S. Patent No. 5,604,531 to Iddan et al., and to embodiments 
described in published application WO01/65995 to Glukhovsky et al., both of which 
are incorporated herein by reference in their entirety, but in alternate embodiments 

20 there may be other types of imaging devices. In one embodiment, an imaging 
device may include more than one image sensor. The image sensors may, for 
example be arranged at either end of device 40, or at the same end of the device, 
in slightly different positions or different angles. A device which includes a plurality 
of image sensors, is described for example, in International Publication Number 

25 WO 02/054932 entitled "System and Method for Wide Field Imaging of Body 
Lumens" which is assigned to the common assignee of the present invention and 
which is hereby incorporated by reference. Each image sensor may capture in vivo 
images and transmit the in vivo images via the transmitter 41 or via separate 
transmitters. The separate in vivo image streams captured may be combined into 

30 a single in vivo image stream based on image comparison between the two 
streams or based on other criteria. 
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In another embodiment of the invention, device 40 may include, for 
example, sensors 43 that may be a sensor other than image sensors such as for 
example, temperature sensor, pressure sensor, blood sensor, pH sensor, 
ultrasound sensor, oxygen sensor, other sensor, and/or more than one sensor. 

5 Data sampled from sensor 43 may, for example, be transmitted via transmitter 41 
or via separate transmitters. 

Typically, located outside the patient's body in one or more locations may 
be an image receiver 12, a data processor 14, and an image monitor 18. Image 
receiver 12 may typically include an antenna or antenna array 15 and an image 

10 receiver storage unit 16. Data processor 14 may include, for example, a processor 
19 and a storage unit 21. In one embodiment of the present invention, storage unit 
21 may include, for example, an image database 22. Image monitor 18 may 
display, inter alia, images transmitted by, for example, device 40. Typically, data 
processor 14 and monitor 18 may be part of a personal computer or workstation, 

15 which includes standard components such as processor 19, a memory, a disk 
drive, and input-output devices, although alternate configurations are possible. 
Data processor 14 may typically, as part of its functionality, acts as a controller 
controlling the display of the images. Image monitor 18 may typically be a 
conventional video display, but may, in addition, be any other device capable of 

20 providing images or other data and may be of any size monitor including small 
palm type monitors or large projection size monitors. The image monitor 18 
presents the image data, typically in the form of still and streaming image frames, 
and in addition may present other information. In an exemplary embodiment, the 
various categories of information are displayed in windows. Other displaying 

25 formats may be used. Multiple monitors may be used to display image and other 
data. The images captured by the imaging system may be of any suitable shape 
including for example circular, square, rectangular, or hexagonal, oval etc. 

In operation, image sensor 46 may capture images and may send data, for 
example, image data to transmitter 41, which may transmit data to receiver 12 

30 using, for example, electromagnetic radio waves. Other types of data other than 
image data may be transmitted. Receiver 12 may transfer the data to receiver 
storage unit 16. Image data stored in storage unit 16 may be transferred to the 
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data processor 14 and/or the data processor storage unit 21. For example, the 
receiver 12 or receiver storage unit 16 may be tal<en off the patient's body and 
may be connected to the personal computer or worl^station that may include the 
data processor 14 via a standard data linl<, e.g., a serial, parallel, USB, or wireless 

5 interface of known construction. The data, for example, image data may then be 
transferred from the receiver storage unit 16 to a database 22 that may be included 
in data processor storage unit 21 . Typically, the image stream is stored as a series 
of images in the image database 22, which may be implemented in a variety of 
known manners. Data processor 14, including dedicated software may analyze the 

10 data and provide the analyzed data to the image monitor 18, where a user may 
view the image data and/or other data. In one embodiment of the invention, more 
than one image stream may be captured from more than one image sensor. In this 
embodiment, data processor 14 may be used to combine the separate image 
streams into one image stream. The combining of the two or more image streams 

15 may be for example, based on comparing images from each Image stream, based 
on time of image capture, or based on other criteria or more than one criterion. 
Other configurations may allow for real time viewing where, for example, device 40 
or receiver 12 may transmit data in real time to data processor 14 for real time 
viewing on monitor 1 8. 

20 Typically, according to one embodiment, an in-vivo imaging device, such 

as for example device 40, collects a series of still images as it may traverse the Gl 
tract. In one embodiment of the present invention, device 40 may collect a large 
volume of data, as it may take several hours to traverse the Gl tract, and may 
record images at a rate of. for example, two images every second, resulting in the 

25 recordation of, for example, thousands of images. The image recordation rate (or 
frame capture rate) and recordation duration may be varied. 

Typically, the data captured and transmitted by the device 40 may be 
digital color image data, although in alternate embodiments other image fomnats 
may be used and data other than and/or in addition to image data may be 

30 transmitted. In an exemplary embodiment, each frame of image data may include, 
for example, 256 rows of 256 pixels, each pixel may include, for example, bytes for 
color and brightness, according to known methods. In other embodiments, image 



6 



wo 2005/062715 



PCT/IL2004/001181 



data may include more a less pixels and each pixel may include other information 
or other number of bytes. Other suitable methods of recording image data may be 
used. Typically, images may be stored sequentially in, for example, data processor 
storage unit 21. In other embodiments, image data and/or other data may be 
stored in receiver storage 16. 

Typically, data processor storage unit 21 may store a series of images 
captured by device 40, for example, as it moves through a patient's Gl tract. The 
series of images may be combined consecutively to form an image stream. When 
viewing the image stream, the user may be typically presented with one or more 
windows on monitor 18; in alternate embodiments multiple windows need not be 
used and only the image stream may be displayed. Reference is now made to Fig. 
IB showing a schematic diagram of display 18. In an embodiment where multiple 
windows are provided, for example, an image window 50 may display the image 
stream or still portions of that image stream. A sensor window 53, may display a 
concurrent stream of data from sensors other than Image sensors, for example, 
ultrasound, pressure, temperature, blood, oxygen, and/or a pH sensor. Other 
sensors may be used. A location window 52 may display information on the 
location of the imaging device, for example, device 40, at the time of capturing the 
image frames being displayed. A control window 54 may include buttons or other 
controls that may be used to alter the display of the image; for example, stop, play, 
pause, capture image, step, fast-forward, rewind, or other. Such controls may be 
activated by, for example, a pointing device such as a mouse or trackball. 
Typically, the Image stream may be frozen to view one frame or one set of frames, 
speeded up, or reversed; sections may be skipped; or any other method for 
viewing an image may be applied to the image stream. Other windows may be 
included. 

While the following discussion relates to the case where data from a 
device 40 is stored for later use, the system and method of the present invention 
may be used with systems allowing for real time viewing of image data. 

Furthermore, while typically the components accepting, processing and 
displaying the image data are contained within a workstation system or PC, other 
systems may be used, and other (e.g., distributed) components may perform such 
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image accepting, processing and displaying. For example, a large projection type 
monitor may be used to view a plurality of images substantially simultaneously. 

In one embodiment of the present invention, device 40 may iiave more 
than one imager. Multiple image streams may result from multiple imagers being 

5 in the same device, or possibly by splitting one image stream into multiple images 
streams. Combining of image streams may be perfonned by for example a 
processing device, e.g. processor 19. Reference is now made to Fig. 6, showing 
an exemplary flow chart describing a method for combining multiple image streams 
into a single image stream. Each image sensor may capture in vivo images and 

10 transmit the in vivo images via the transmitter 41 or via separate transmitters. 
Alternately, one image stream may be split (e.g., every other image may be placed 
into one of two image streams, etc.). The separate image streams obtained from 
the multiple image sensors may be stored (step 610). Comparison may be made 
between the frames captured from the separate image streams (step 620). In 

15 other embodiments of the invention, the separate image streams may not be 
stored in their entirety before comparison is made. Based on the comparison, 
frames from each of the separate image streams may be slotted into a single 
image stream (step 630). Other methods besides or in addition to comparing 
frames may be used to combine multiple image streams. The separate image 

20 streams may be combined such that frames displaying similar scenery are slotted 
in close proximity to each other. As such, the multiple image streams may be 
displayed in a variable order and/or manner. Other suitable criteria may be used 
as well. 

In one embodiment, the imaging window (e.g. monitor 18) may display a 
25 multi-frame stream where multiple images are displayed substantially 
simultaneously in each time slot, thereby reducing the viewing time. In one 
embodiment, processor 19 may determine spatial positioning of each of the frames 
displayed substantially simultaneously. For example, in one embodiment, the 
original image stream may be divided into, for example, separate image streams 
30 displayed substantially simultaneously in single time slots. Various methods may 
be used to separate an original image stream into a plurality of image streams to 
be streamed substantially simultaneously. In one example, images from the 
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original image stream may be divided into, for example, 16 subset image streams 
such that, for example, during the first time slot, the first 16 images of the original 
image stream may be displayed substantially simultaneously, and in the second 
time slot the next 16 images of the original images stream may be displayed 
substantially simultaneously, etc. Other number of image streams may be used 
and some frames in the original stream may be skipped. In altemate 
embodiments, the images may be separated, for example, placed In different files 
or memory blocks in processor 14. In one embodiment, each resulting image 
stream may include a separate subset of images from the original image stream; in 
alternate embodiments, the images from each resulting image stream may overlap. 
In another embodiment of the invention, images of the original image stream may 
be divided between the separate images streams in alternate manners that may be 
suitable. Typically, the spatial positioning and/or order of the frames, for example, 
the frames obtained from corresponding separate image streams may vary for 
each time slot. 

In one embodiment, images from the original image stream may be simply 
directed to the proper screen position at viewing time, and the Image stream may, 
for example, not actually be separated in separate image streams. In such 
embodiments, a different number of images may be displayed substantially 
simultaneously for every time slot. Typically, the spatial positioning of one or more 
frames displayed substantially simultaneously may be in a variable order, e.g. an 
order other than the chronological order that the frames were captured. In one 
embodiment, each time slot may display all image frames captured from a 
specified region. For example, the first time slot may include all frames captured in 
the esophagus, the second time slot may display all images captured in the 
stomach, etc. In an alternate embodiment the same number of image frames may 
be displayed in each time frame. 

Fig. 2 depicts a portion of a display according to an embodiment of the 
present invention. The display 200 may include, for example, 16 image frames 
displayed substantially simultaneously in a single time slot (210A-240D). In an 
exemplary embodiment, the windows or viewing areas, 21 OA, 21 OB etc., may be 
close together, with a minimum of blank or black space between the frames, and 
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typically may be displayed in matrix form, to allow a viewer to see the entirety of the 
frames without substantially moving his eyes. Other suitable spatial arrangements 
may be used. 

The spatial order or position of the image frames displayed in a single time 
5 slot may be adjusted and/or varied based on for example, computations made by 
processor 19. The spatial order of the image frames displayed substantially 
simultaneously may correspond to the chronological order in which the images may 
have been captured, may correspond to an alternate predetermined criteria, to 
more than one predetermined criteria, or other criteria of interest. In one 
10 embodiment of the invention the spatial order or position of the frames displayed 
may be based on a predetermined criteria, and/or criteria of interest. In one 
embodiment of the invention, the image frames displayed in a single time slot may 
be spatially positioned based on a predetermined criterion of, for example color 
variation. Other criteria, such as focusing, lighting, blood detection, etc. may be 
15 used. 

An exemplary algorithm is listed in Fig. 3. Refening to block 318, a set of 
image frames to be displayed substantially simultaneously in a single time slot may 
be selected. In some embodiments of the present invention, the set of Images to 
be displayed substantially simultaneously in a single time slot may be selected 

20 based on for example predetermined criteria. For example, images displaying 
specified features e.g. color scheme, may be selected to be displayed substantially 
simultaneously during a single time slot. Other suitable criterion may be used, e.g. 
focusing, lighting, blood detection, shape features, etc. Normalization or 
adjustment of lighting may be perfomned on the Image frames to be displayed 

25 (block 320). This may be perfomned to reduce any color variation due to 
shadowing, variation in lighting intensity, or variation in lighting and image sensor 
properi:ies when, for example, image frames may be captured from more than one 
image sensor. Subsequently, a reference image may be generated based on the 
image frames to be displayed (330). In one embodiment of the invention, the 

30 reference image frame may be, for example, an image representing the average or 
sum of the image frames to be displayed substantially simultaneously in a single 
time slot. In another embodiment of the present invention, the reference image 
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may be, for example, a selected image frame that may have some known 
properties, an image selected or generated from a previous time slot, or other 
suitable images. In one embodiment of the invention, the reference image may 
represent a typically healthy looking tissue. In step 340, the image frames being 

5 displayed in the time slot, may be compared to the generated image. The 
comparison may be based on comparing sections of images or may be based on 
comparing whole images. In one example, images displaying pathology may show 
a high degree of variation with respect to the reference image, for example when 
the reference image may represent a typically healthy tissue. A score may be 

10 given to each image based on the comparison performed (block 350). In another 
embodiment of the invention, image frames to be displayed in a single time slot 
may be compared to more than one reference image and receive more than one 
score. The spatial position of the image frames displayed and/or other suitable 
parameters, e.g. size may be based on the scores they were assigned (360). For 

15 example, according to one embodiment pictures having similar scores may be 
positioned where a viewer may typically draw his/her eyes first, e.g., at a corner of 
the display. 

Other post processing methods may be included as well to differentiate 
between images that may display, for example, pathology and images that may 

20 display, for example, bubbles, content, or other variations through the Gl tract, 
which may not be associated with pathology or other criteria of interest. In another 
embodiment of the present invention, the generated image may represent an 
average image generated from the entire original image stream, from a 
predetermined section of the original image stream, or from image streams taken 

25 from a pool of volunteers. In another example the generated and/or pre-selected 
image may represent a pathology or specific information that is being sought in the 
plurality of image streams. As such, images with the lowest degree of variation 
with respect to the reference image may be representative of the images of 
interest Other algorithms may be used as well. As an example, images detected 

30 as displaying blood may be a criterion of interest. Algorithms for blood detection 
have been previously described in US Application Number 10/097,096 filed on 
March 14, 2002, assigned to the common assignee of the present invention and 
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hereby incorporated by reference. In another embodiment of the invention other 
criteria besides or together with color variation may be used. For example, shape 
variation may be used as a criterion to help identify polyps in the Gl tract. Variation 
above or below a certain threshold based on a selected or generated image may 

5 be used. The threshold may be adjusted to increase or decrease sensitivity. Other 
suitable criteria may be used as well. 

In yet another embodiment of the invention, the criteria of interest that may 
determine the spatial order of the image frames in each time slot, and the number 
of images displayed per time slot may be based on readings concurrently or 

10 substantially concurrently obtained from sensors other than image sensor, for 
example the spatial positioning of image frames displayed substantially 
simultaneously in a single time slot may be based on reading obtained from one or 
more non-image sensors mentioned herein. 

in some embodiments of the present invention, in-vivo sensors readings 

15 may be used as input, for example, to processor 19, to sort spatial positioning of 
frames in each time slot. Other suitable methods of determining spatial positioning 
may be used. Reference is now made to Fig. 4 showing a flow chart for sorting 
image frames displayed substantially simultaneously in a single time slot. In step 
510, images to be displayed substantially simultaneously may be selected. 

20 Corresponding sensor readings from one or more non-image sensors (e.g. 
temperature sensor, pressure sensor, blood sensor, pH sensor, ultrasound sensor, 
oxygen sensor, etc.) may be selected (step 520). For example, temperature 
readings recorded during the time of capturing the image frames selected in step 
510 may be used. Images selected in step 510 may be sorted (step 530) based on 

25 the corresponding sensor data selected in step 520, in for example ascending or 
descending order Other methods of sorting may be implemented. In one 
embodiment of the present invention, sorting may be based on reading of more 
than one sensor and may be based on image analysis, or image analysis as well 
as sensor readings. In another embodiment of the present invention, images may 

30 also be sorted based on for example a one or zero output of a sensor, so that all 
images corresponding to a one output may be grouped together and all images 
corresponding to a zero output may be grouped together. Images corresponding 
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to sensor readings passing a defined tlireshold may be tagged (540). Sucli 
images may be emphasized during display as may be described herein. 

Typically, increasing the number of image frames displayed substantially 
simultaneously in a single time slot, may allow for a processing method such as the 

5 examples described in Fig. 3 and Fig. 4 to single out, for example, a relatively small 
number of frames showing pathology from, for example, a relatively large number 
of frames showing typically healthy looking tissue. Increasing the number of 
images displayed in a single time slot may decrease the emphasis on small 
variation in, for example, healthy tissue while possibly increasing the emphasis on 

10 large variations that may appear in for example, pathological tissues. In addition, 
increasing the number of images displayed in a single time slot may decrease the 
viewing time and/or the length of the displayed image stream. 

In one embodiment of the invention, image frames that may have showed, 
for example, a high degree of variation may be positioned, for example, on monitor 

15 18, in a location where the user may mostly likely glance first, and image frames 
that may have shown, for example, a low degree of variation may be positioned, for 
example, where the user may most likely to glance last. In one embodiment of the 
present invention, it may be assumed that the user may scan the matrix of images 
starting at the upper left hand corner. In this embodiment, image frames that may 

20 be of interest, for example showing a high degree of variation with respect to 
frames, for example, representing healthy looking tissue, may be positioned in the 
upper left hand corner, starting with for example frame 21 OA (Fig. 2). Image 
frames that may be of least interest, for example, images showing a low degree of 
variation with respect to images representing healthy looking tissue, may be 

25 positioned in the lower right hand corner, ending with for example with frame 240D 
(Fig. 2). Frames may be arranged by descending order of degree of variation. The 

progression from frame 210A to frame 240D may be by row (e.g. 210A, 210B 

220A, 220B, ...230A, ..240D), by column (e.g. 210A, 220A, 210B, 220B, 
...210C, ...240D), or radially (e.g. 21 OA, 21 OB, 220B, 220A, 21 OC, 220C, 230C, 

30 230B, ...240D). Other suitable progression series may be used. In another 
embodiment of the present invention, image frames that may be of interest, for 
example showing a high degree of variation, may be positioned in the center of the 
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image window, at a height, for example, 2/3 above the bottom, while images 
showing a low degree of variation may be positioned in the periphery of the image 
window. The progression from image frames showing a high degree of variation to 
image frames showing a low degree of variation may be a spiral shaped. Spiral 

5 progressions may be clockwise spiral progressions, or counter-clockwise 
progressions. Other suitable criteria may be used besides degree of variation. 

Other suitable image frame arrangements may be used as well. For 
example, image frames in single time slot that meet a particular criteria of interest 
above a defined threshold may in one embodiment of the present invention, be 

10 grouped together for emphasis, by for example, fusion at the borders to form a 
single entity, or emphasized by, for example, highlighting, inserting a border around 
grouped image frames, etc. For example, in Fig. 2 image frames 21 OA and 21 OB 
may be emphasized by border 299. Other number of frames or types of grouping 
may be used. 

15 In one embodiment of the invention, images displayed substantially 

simultaneously may be of different sizes or shapes. In one example, specific 
images from the group of images displayed substantially simultaneously may be 
enlarged or displayed in a larger size so as to emphasize them. Reference is now 
made to Fig. 5 showing as an example, 46 image frames displayed substantially 

20 simultaneously. In one embodiment of the invention, image frames having for 
example, the highest degree of variation according to a specified criteria are placed 
in, for example the center (with respect to, for example, the horizontal positioning), 
at a height 2/3 above the bottom. In one example, images with variation above a 
predetermined threshold may be grouped together and displayed in larger frames 

25 such as frames 401, 402, 403, 404, and 405, 406. Other images below the 
specified threshold, for example, might be displayed in smaller frames such as, for 
example, frames 407 to 446. The sensitivity of this grouping may be adjusted by 
increasing or decreasing the defined threshold. The threshold may be adjusted by 
the user or for example by an algorithm adjusting the threshold based on a defined 

30 maximum and minimum number of large frames to be displayed concurrently in a 
single time slot along with other criteria. 



14 



wo 2005/062715 



PCT/IL2004/001181 



In another embodiment of the invention, the larger frames may be 
positioned at the top left comer. In yet another embodiment of the present 
invention different size frames may be positioned in other suitable an-angement. 
As such, a large number of images may be reviewed by a user, such as a health 

5 professional, while only emphasizing the typically few images most likely to contain 
information of interest. The typically large majority of images less likely to contain 
information of interest may be reviewed at a quick glance. In one embodiment of 
the invention, a user may pause the image streaming when identifying an image of 
particular interest. As such the user may examine more carefully the image or 

10 images of interest and the images taken around the vicinity of the image of interest. 
The order that the images may be shown substantially simultaneously may be 
altered to show images for example in a chronological order. Other suitable 
features may be used as well to aid in the analysis of the images obtained. 

In one embodiment of the invention, lighter images may be in general 

15 positioned in the upper part of the image window 50, (Fig. 1B) while in general 
darker images may be positioned in the lower end of image window 50. As such 
the continuity between subsequent time slots may be enhanced for easier viewing 
of a stream. In other embodiments of the present invention, other suitable 
parameters may be used for determining the order of the image frame. 

20 Typically, if normally a single-frame stream may be displayed at a certain 

rate, a multi-frame stream may be displayed at a slower rate. For example if 
typically a single-frame stream may be displayed at 20 frames per second, a multi- 
frame stream may be displayed for example at 5 or less frames per second. In 
such a case the overall viewing time may be up to four times less than the original 

25 single-frame stream viewing time. Other suitable viewing rates and ratios between 
single-frame stream and multi-frame stream viewing time may be used. In one 
embodiment of the invention, the viewing rate or other viewing parameters of the 
multi-frame stream may be adjusted based on the content of the frames displayed 
in each time slot and based on the transition of sequential time slots. For example, 

30 time slots including frames that may meet a criterion of interest may be displayed 
at a slower rate than time slots that may not include frames that meet a criterion of 
interest. In another example, the viewing rate may be reduced where there may be 
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a large variation in the color content in the transition between neighboring time 
slots. In yet another embodiment of the invention, a user may adjust the frame rate 
as well as the number of frames displayed substantially simultaneously. 

Reference is made to Fig. 7 showing an exemplary flow chart for adjusting 
the stream rate of the multi-frame stream according to one embodiment of the 
present invention. In one example, a set of frames to be displayed in a single time 
slot may be examined (step 710). If one or more frames meet a predetemnined 
criterion (step 720), such as a criterion of interest, the stream rate may be adjusted 
(step 750), for example, reduced. This may help to further emphasize frames that 
may be of interest. In another example, the viewing rate may be reduced where 
there may be a large variation in the color content in the transition between 
neighboring time slots. Frames displayed in neighboring time slots may be 
compared (step 730). If there is a large variation in frames displayed in 
neighboring time slots, for example a large variation in color content or texture, the 
stream rate may be reduced (step 740). If there is no significant variation and no 
frames meeting a criterion of interest, the stream rate may be maintained (step 
760). In yet another embodiment of the invention, a user may adjust the frame rate 
as well as the number of frames displayed substantially simultaneously. 

In an exemplary embodiment, the user may switch modes, for example 
between viewing the images as one stream and viewing the images as a plurality 
of streams using a control such as a keystroke, mouse, or on-screen button. The 
user may control the plurality of streams in a manner similar to the control of a 
single stream, for example by using on screen controls. In an alternate 
embodiment, only one mode may be offered to the user. In yet an alternate 
embodiment of the invention, a user may select a mode of only viewing images 
meeting a defined criterion of interest. 

In some embodiments, each image may be displayed with different post- 
processing. For example, one image may be subject to certain filtering or 
manipulation (e.g., red or green filtering, contrast enhancement, brightness 
alteration) and the other image may be subject to different or no filtering or 
manipulation. In one embodiment two or more images in a plurality of streams 
displayed substantially simultaneously may be fused together, for example, at the 
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borders and may be displayed as a single entity. As such, a user may comfortably 
and concurrently incorporate information shown in each of the images while 
avoiding the distraction that may be caused by the typically sharp contrast between 
connecting edges or between the images and the background color which may 

5 appear between the images when the images are spaced apart. According to 
some embodiments fusing borders between independent images may be 
accomplished by, for example, one or more post processing algorithms known in 
the art, including but not limited to, smoothing convolution, mirroring, overlapping, 
linear or non-linear fade-out fade-in, truncation, linear shape distortion, non-linear 

10 shape distortion, normalization or intensity, or other suitable post-processing. 

It will be appreciated by persons skilled in the art that the present invention 
is not limited by what has been particularly shown and described hereinabove. 
Rather the scope of the invention is defined by the claims that follow: 
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